Q80903.APP.CWB 
April 16, 2004 

APPARATUS AND METHOD FOR DETERMINING 4X4 INTRA 
LUMINANCE PREDICTION MODE 

BACKGROUND OF THE INVENTION 
[01] This application claims the priority of Korean Patent Application No. 
2003-25528, filed on April 22, 2003, in the Korean Intellectual Property 
Office, the disclosure of which is incorporated herein in its entirety by 
reference. 

1. Field of the Invention 

[02] The present invention relates to a codec for video data, and more 
particularly, to an apparatus and method for determining a prediction mode, 
which are used in a video codec. 

2. Description of the Related Art 

[03] Broadcast television and home entertainment have been revolutionized 
by the advent of digital TV and DVD-video. These applications and many 
more were made possible by the standardization of video compression 
technology. The next standard in the MPEG series, MlPEG4-visual, is 
enabling a new generation of internet-based video applications whilst the ITU- 
T H.263 standard for video compression is now widely used in 
videoconferencing systems. 
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[04] MPEG4-visual and H.263 are standards that are based on video 
compression. The groups responsible for these standards, the Motion Picture 
Experts Group and the Video Coding Experts Group (MPEG and VCEG) are 
in the final stages of developing a new standard that promises to significantly 
outperform MPEG4 and H.263, providing better compression of video images 
together with a range of features supporting high-quality, low bit-rate 
streaming video. 

[05] After finalizing the original H.263 standard, the ITU-T Video Coding 
Experts Group (VCEG) started work on two further development areas: short- 
term efforts to add extra features to H.263 (resulting in Version 2 of the 
standard) and long-term efforts to develop a new standard for low bit-rate 
visual communications. The long-term effort led to the draft H.26L standard, 
offering significantly better video compression efficiency than previous ITU-T 
standards. The ISO Motion Picture Experts Group (MPEG) recognized the 
potential benefits of H.26L and the Joint Video Team (JVT) was formed, 
including experts from MPEG and VCEG. The main task of the JVT is to 
develop the draft H.26 model into a full International Standard. In fact, the 
outcome will be two identical standards: ISO MPEG4 Part 10 of MPEG4 and 
ITU-T H.264. The title of the new standard is Advanced Video Coding 
(AVC); however, it is widely known by its old working title, H.264. 
[06] FIG. 1 is a block diagram of an H.264 encoder. 

[07] The H.264 encoder includes a prediction unit 110, a transform and 
quantization unit 120, and an entropy coding unit 130. 
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[08] The prediction unit 110 performs inter prediction and intra prediction. 
Inter prediction is prediction for a block of a present picture using a reference 
picture which is subjected to decoding and deblocking filtering and stored in a 
buffer. That is, inter prediction is prediction using several pictures of data. To 
perform such inter prediction, the prediction unit 110 includes a motion 
estimator 111 and a motion compensator 112. Intra prediction predicts a 
predetermined block on a decoded picture using pixel data of its adjacent 
blocks. 

[09] The transform and quantization unit 120 transforms and quantizes a 
prediction sample obtained from the prediction unit 110. The entropy coding 
unit 130 encodes the quantized result into an H.264 bit stream according to a 
predetermined format. 

[10] FIG. 2 is a block diagram of an H.264 decoder. 

[11] The H.264 decoder receives and entropy-decodes a bit stream encoded 

by the H.264 encoder, performs dequantization and inverse-transformation of 

the decoded result, and then decodes the result using reference picture 

information subjected to motion compensation or intra prediction. 

[12] FIG. 3 shows a luminance block P to be predicted and its adjacent 

blocks to be used for prediction of the luminance block P. 

[13] If blocks or macroblocks on a picture have been encoded in an intra 

mode, a block P (310) to be predicted can be predicted using its adjacent 

decoded blocks A through L. Prediction is performed for chrominance blocks 

Cb and Cr as well as for luminance (hereinafter, briefly referred to as "luma") 
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blocks, however, for convenience of descriptions, prediction for only luma 
blocks is described in the present invention. The luma prediction block P (310) 
is a 16x16 block consisting of several 4x4 blocks. In FIG. 3, small letters a 
through p are 4x4 blocks to be predicted and capital letters A, B, C, D and I, J, 
K, L are adjacent blocks to be used for predictions of the 4x4 blocks a through 

P' 

[14] Intra prediction is classified into 4x4 prediction and 16x16 prediction 
according to the size of a block to be predicted. 4x4 prediction has nine 
modes and 16x16 prediction has four modes, according to different directions 
of predictions. When the block P (310) is predicted, prediction samples are 
obtained in the nine 4x4 prediction modes according to the different directions 
of predictions, using pixel values of the blocks (A, B, C, D and I, J, K, L) 
adjacent to the 4x4 blocks to be predicted. 

[15] FIG. 4 is a table listing types of intra 4x4 luminance prediction modes. 
[16] Referring to FIG. 4, the 4x4 intra luma prediction modes include a 
vertical mode, a horizontal mode, a DC mode, a diagonal_down_left mode, a 
diagonal_down_right mode, a vertical_right mode, a horizontal_down mode, a 
vertical_left mode, and a horizontal_up mode. Directions in which predictions 

are performed in the respective prediction modes will be described with 

j 

reference to FIG. 5. Predictions of blocks in the respective modes will be 
described with reference to FIGS. 6 A through 61. 

[17] FIG. 5 shows nine prediction directions for H.264 4x4 intra luminance 
prediction. 
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[18] Referring to FIG. 5, a block is predicted in a vertical direction, a 
horizontal direction, a diagonal direction, etc., each corresponding to a mode 
type. 

[19] FIGS. 6 A through 61 are views for describing predictions according to 
the 4x4 intra luminance prediction modes. 

[20] For example, in a mode 0 (vertical mode), 4x4 blocks a, e, i, and/n are 
predicted using a pixel value of a block A; 4x4 blocks b, f, j, and n are 
predicted using a pixel value of a block B; 4x4 blocks c, g, k, and o are 
predicted using a pixel value of a block C; and 4x4 blocks d, h, 1, and p are 
predicted using a pixel value of a block D. Predictions according to other 
modes are disclosed in detail in the H.264 standard. 

[21] When H.264 encoding is performed, an optimal mode among the 4x4 
intra luma prediction modes is selected and prediction is performed in the 
optimal mode. Compression efficiency is different according to the mode in 
which luma prediction for a 4x4 block is performed. To select an optimal 
mode, a block is predicted in all modes, costs are calculated using a 
predetermined cost function, and a mode with a smallest cost is selected as the 
optimal mode. Accordingly, since a block to be predicted shquld be predicted 

in all of the nine modes and costs should be calculated respectively in the nine 

j 

modes, an encoder becomes complicated. 

SUMMARY OF THE INVENTION 
[22] The present invention provides an apparatus and method for 
determining a 4x4 intra luminance prediction mode, in which only a few 
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modes, not all of nine modes are used to determine an optimal 4x4 intra 
luminance prediction mode. 

[23] According to an aspect of the present invention, there is provided a 
prediction mode determination method comprising: (a) grouping a plurality of 
predetermined prediction modes into a predetermined number of groups; (b) 
selecting one prediction mode for each of the groups, respectively, performing 
predictions for a predetermined block in the selected prediction modes, and 
calculating prediction errors in the selected prediction modes; and (c) 
performing predictions for the predetermined block in other prediction modes 
belonging to a group including a prediction mode with a smallest prediction 
error among the calculated prediction errors, calculating prediction errors in 
the other prediction modes, and deciding a prediction mode according to the 
calculated prediction errors. 

[24] According to another aspect of the present invention, there is provided 
a prediction mode determination method comprising: (a) performing 
predictions for a predetermined block in a vertical prediction mode 
corresponding to a vertical direction, a horizontal prediction mode 
corresponding to a horizontal direction, and a DC prediction mode 
corresponding to a current location, and calculating prediction errors in the 
respective modes; and (b) performing predictions for the predetermined block 
in prediction modes corresponding to prediction directions adjacent to a 
prediction direction of a prediction mode with a smallest prediction error 
among the calculated prediction errors, calculating prediction errors in the 
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prediction modes, and selecting a prediction mode with a smallest prediction 
error among the calculated prediction errors. 

[25] According to still another aspect of the present invention, there is 
provided a prediction mode determination apparatus comprising: a prediction 
unit, which performs prediction for a predetermined block in a predetermined 
prediction mode, and outputs a prediction sample; a prediction error calculator 
which calculates a prediction error for the prediction sample; and a prediction 
error comparator, which compares prediction errors received from the 
prediction error calculator to each other, selects a prediction mode with a 
smallest prediction error according to the compared result, and outputs the 
selected prediction mode. 

[26] According to a further aspect of the present invention, there is provided 
a computer readable medium having embodied thereon a computer program 
for the prediction mode determination method of claim 1. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[27] The above and other features and advantages of the present invention 
will become more apparent by describing in detail exemplary embodiments 
thereof with reference to the attached drawings in which: 
[28] FIG. 1 is a block diagram of an H.264 encoder; 
[29] FIG. 2 is a block diagram of an H.264 decoder; 

[30] FIG. 3 shows a luminance block P to be predicted and its adjacent 

blocks to be used for prediction of the luminance block P; 

[31] FIG. 4 is a table listing types of intra 4x4 luminance prediction modes; 
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[32] FIG. 5 shows nine prediction directions used for 4x4 intra luminance 
prediction according to H.264; 

[33] FIGS. 6A through 61 are views for describing predictions according to 
the 4x4 intra luminance prediction modes; 

[34] FIG. 7A is a view that partitions prediction directions corresponding to 

the 4x4 intra luminance prediction modes into three groups; 

[35] FIG. 7B is a flowchart illustrating a prediction mode determination 

method according to an embodiment of the present invention; 

[36] FIG. 8 is a detailed flowchart illustrating the prediction mode 

determination method according to the embodiment of the present invention; 

[37] FIG. 9A is a view that partitions prediction directions corresponding to 

prediction modes into three groups, according to another embodiment of the 

present invention; 

[38] FIG. 9B is a flowchart illustrating a prediction mode determination 
method according to another embodiment of the present invention; 
[39] FIGS. 10A and 10B are a detailed flowchart illustrating the prediction 
mode determination method according to other embodiments of the present 
invention; and 

[40] FIG. 11 is a block diagram of a prediction mode determination 
apparatus according to an embodiment of the present invention. 
DETAILED DESCRIPTION OF THE ILLUSTRATIVE. NON-LIMITING 
EMBODIMENTS OF THE INVENTION 
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[41] Hereinafter, embodiment of the present invention will be described in 
detail with reference to the appended drawings. 

[42] FIG. 7A is a view that partitions prediction directions corresponding to 
4x4 intra luminance (hereinafter, referred to as "luma") prediction modes into 
three groups. 

[43] In the present invention, nine prediction modes for 4x4 intra. luma 
prediction are grouped into groups of three and predictions are performed on 
each of the groups. In more detail, a mode 0 (vertical direction) and its 
adjacent modes 5 and 7 are grouped into a group 1 (701), a mode 1 (horizontal 
direction) and its adjacent modes 6 and 8 are grouped into a group 2 (702), and 
a mode 2 (DC) and the remaining modes 3 and 4 (diagonal direction) are 
grouped into a group 3 (703). Then, predictions are performed in 
representative modes 0, 1, and 2 of the respective groups, costs are calculated 
in the modes 0, 1, and 2, predictions are performed in other modes within a 
group including a mode selected according to calculated costs, and costs are 
calculated in the other modes. 

[44] FIG. 7B is a flowchart illustrating a prediction mode determination 
method according to an embodiment of the present invention. r 

[45] First, predictions for a predetermined block are performed in the 

j 

modes 0, 1, and 2 and costs CO, CI, and C2 are calculated in the respective 
modes (step S710). The cost is a residue obtained by subtracting an original 
pixel value of a prediction target sample (predetermined block) from a 
predicted pixel value of the prediction target sample. The costs are calculated 
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by applying an SSD function, an SAT function, or an SATD function (they 
will be described later) for the prediction target sample. A method for 
calculating the costs will be described later. Then, it is determined which cost 
among the costs CO, CI, and C2 is smallest (step 711). According to the 
determined result, predictions for the predetermined block are performed in 
the modes 5 and 7 adjacent to the mode 0 and costs C5 and C7 are calculated 
(step S712), predictions for the predetermined block are performed in the 
modes 6 and 8 adjacent to the mode 1 and costs C6 and C8 are calculated (step 
S713), or predictions for the predetermined block are performed in the modes 
3 and 4 adjacent to the mode 2 and costs C3 and C4 are calculated (step S714). 
Then, the cost values calculated in steps S712, S713, or S714 are compared to 
one another and a mode with a smallest cost is selected (step S715). That is, 
min(CO, C5, C7), min(Cl, C6, C8), or min(C2, C3, C4) is calculated, so that a 
mode with a smallest cost is selected. 

[46] FIG. 8 is a detailed flowchart illustrating the prediction mode 
determination method according to the embodiment of the present invention. 
[47] Predictions for a 4x4 block are performed in mode 0 (vertical mode), 
mode 1 (horizontal mode), and mode 2 (DC mode) (steps §802, S804, and 
S806). Accordingly, prediction samples for the 4x4 block are obtained. Then, 
prediction costs CO, CI, and C2 for the prediction samples are calculated using 
a cost function (steps S808, S810, S812). The prediction costs CO, CI, and C2 
are residual values (that is, prediction errors) obtained by subtracting original 
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pixel values of the prediction samples from predicted pixel value of the 
prediction samples. 

[48] Methods for calculating the costs are various. The cost function 
includes an SAD (sum of absolute value) function, an SATD (sum of absolute 
transformed difference) function, an SSD (sum of squared difference) function, 
a MAD (mean of absolute difference) function, a Lagrange function, and the 
like. The SAD function sums the absolute values of the prediction errors 
(residues) for a 4x4 block. The SATD function sums the absolute values of 
coefficients obtained by applying a Hadamard Transform to the prediction 
errors for a 4x4 block. The SSD function sums the square values of the 
prediction errors for a 4x4 block prediction sample. The MAD function 
calculates the average value of the absolute values of the prediction errors for 
a 4x4 block prediction sample. The Lagrange function is a new cost function 
of adding bit-stream length information to one of the above-mentioned cost 
functions. 

[49] Then, it is determined which cost value among the cost values CO, CI, 
and C2 calculated in modes 0, 1, and 2 is smallest (step S814). Predictions are 
performed in different modes according to the determined yesult. In more 
detail, if the cost value CO of the mode 0 is smallest, predictions are performed 
in modes 5 and 7 adjacent to the mode 0 (steps S816 and S818). If the cost 
value CI of the mode 1 is smallest, predictions are performed in the modes 8 
and 6 adjacent to the mode 1 (steps S820 and step S822). If the cost value C2 
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of the mode 2 is smallest, predictions are performed in the modes 3 and 4 
adjacent to the mode 2 (steps S824 and S826). 

[50] Then, the cost values C5 and C7 of the modes 5 and 7 are calculated 
respectively (steps S828 and S830), the cost values C8 and C6 of the modes 8 
and 6 are calculated respectively (steps S832 and S834), or the cost values C3 
and C4 of the modes 3 and 4 are calculated respectively (steps S836 and S838). 
[51] If it is determined that the cost value CO is smallest in step S814 and 
accordingly steps S816, S818, S828, and S830 are performed, a smallest cost 
value among the calculated cost values CO, C5, and C7 is searched for (step 
S840) and a mode corresponding to the smallest cost value is selected. 
Likewise, if it is determined that the cost value CI is smallest in step S814 and 
accordingly steps S820, S822, S832, and S834 are performed, a smallest cost 
value among the calculated cost values CI, C6, and C8 is searched for (step 
S842) and a mode corresponding to the smallest cost value is selected. Also, 
if it is determined that the cost value C2 is smallest in step S814 and 
accordingly steps S824, S826, S836, and S838 are performed, the smallest 
cost value among the calculated values C2, C3, and C4 is searched for (step 
S844) and a mode corresponding to the smallest cost value is ^elected. 
[52] Therefore, one among the modes 0, 5, and 7 is selected (steps S846, 
S848, S850), one among the modes 1, 6, and 8 is selected (steps S852, S854, 
S856), or one among the modes 2, 3, and 4 is selected (steps S858, S860, 
S862). 



12 



[53] As described above, an optimal prediction mode can be selected by 
performing predictions in only five prediction modes, instead of performing 
predictions in all prediction modes. That is, by partitioning a plurality of 
prediction modes into several groups and performing predictions on each of 
the groups, an optimal mode with a smallest prediction error can be selected 
using lower calculation times. 

[54] FIG. 9 A is a view that partitions prediction modes into three groups, 

according to another embodiment of the present invention. 

[55] To improve prediction performance, according to another embodiment 

of the present invention, each of the groups further includes additional 

adjacent modes. In more detail, the modes 0, 7, and 5 and their adjacent 

modes 3 and 4 are grouped into a group 1 (901); the modes 1, 8, and 6 and 

their adjacent mode 4 are grouped into a group 2 (902); and the modes 2, 3, 

and 4 and their adjacent modes 7, 5, and 6 are grouped into a group 3 (903). 

Then, predictions are performed for each of the newly created groups. 

[56] FIG. 9B is a flowchart illustrating a prediction mode determination 

method according to another embodiment of the present invention. 

[57] First, predictions are performed in modes 0, 1, and 2 and costs CO, CI, 

and C2 in the respective modes are calculated (step S910), as described above. 

j 

Then, it is determined which cost among the costs CO, CI, and C2 is smallest 
(step S911). 

[58] If it is determined that the cost CO is smallest in step S911 and costs 
C5 and C7 in the modes 5 and 7 are calculated in step S912, mode selection is 
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again conducted according to which cost among the costs CO, C5, and C7 is 
smallest (step S915). If the cost C5 is smallest, prediction is performed in the 
mode 4 adjacent to the mode 5, a cost C4 is calculated, and a mode with the 
smaller cost value among the costs C5 and C4 is selected as an optimal mode. 
If the cost C7 is smallest, prediction is performed in the mode 3 adjacent to the 
mode 7, a cost C3 is calculated, and a mode with the smaller cost value ^paong 
the costs C7 and C3 is selected as an optimal mode. Through such process, an 
optimal mode with a smallest cost is selected. 

[59] Likewise, if it is determined that the cost CI is smallest in step S911 
and accordingly costs C6 and C8 in the modes 6 and 8 are calculated in step 
S913, mode selection is again conducted according to which cost among the 
costs CI, C6, and C8 is smallest. If the cost C6 is smallest, prediction is 
performed in the mode 4 adjacent to the mode 6, a cost C4 is calculated, and a 
mode with the smaller cost value among the costs C4 and C6 is selected as an 
optimal mode. 

[60] Also, if it is determined that the cost C2 is smallest in step S911 and 
accordingly costs C3 and C4 in the modes 3 and 4 are calculated, mode 
selection is again conducted according to which cost among tjhe costs C2, C3, 
and C4 is smallest (step S917). For example, if the cost C3 is smallest, 
prediction is performed in the mode 7 adjacent to the mode 3, a cost C7 is 
calculated, and a mode with the smaller cost among the costs C3 and C7 is 
selected as optimal mode. If the cost C4 is smallest, predictions are performed 
in the modes 5 and 6 adjacent to the mode 4, costs C5 and C6 are calculated, 
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and a mode with the smallest cost among the costs C4, C5, and C6 is selected 
as an optimal mode. 

[61] FIGS. 10A and 10B are a detailed flowchart illustrating the prediction 
mode determination method according to another embodiment of the present 
invention. 

[62] Predictions for a 4x4 block are performed in the mode 0 (vertical 
mode), the mode 1 (horizontal mode), and the mode 2 (DC mode) (steps 
S1001, S1002, and S1003), thereby obtaining prediction samples. Then, costs 
CO, CI, and C2 for the prediction samples are calculated using a cost function 
(steps S1004, S1005, and S1006). The cost function may be one among the 
cost functions described above with reference to FIG. 8. 

[63] Then, a mode with a smallest cost value among the calculated cost 
values CO, CI, and C2 is selected (step S1007). If the cost value CO of the 
mode 0 is smallest, predictions are performed in the modes 5 and 7 adjacent to 
the mode 0 (steps S1008 and S1009). If the cost value CI of the mode 1 is 
smallest, predictions are performed in the modes 8 and 6 adjacent to the mode 
1 (steps S1010 and S1011). Also, if the cost value C2 of the mode 2 is 
smallest, prediction is performed in the mode 3 adjacent to tjie mode 2 (step 
S1013). 

[64] Then, the cost values C5 and C7 in the modes 5 and 7 are calculated 
respectively (steps S1014 and S1015), the cost values C8 and C6 in the modes 
6 and 8 are calculated respectively (steps S1016 and S1017), or the cost values 
C3 and C4 in the modes 3 and 4 are calculated respectively (steps S1018 and 
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S1019). The above steps are the same as those described above with reference 
to FIG. 8. 

[65] If steps S1014 and S1015 are performed, a smallest cost value among 
the costs CO, C5, and C7 is searched for (step S1020). If the cost CO is 
smallest, the mode 0 is selected (step S1021). If the cost C5 is smallest, 
prediction is performed in the mode 4 adjacent to the mode 5 (step S1022). 
Then, a cost C4 is calculated (step S1023) and the cost C4 is compared with 
the cost C5 (step 1024). If the cost C5 is smaller than the cost C4, the mode 5 
is selected (step S1025). If the cost C4 is smaller than the cost C5, the mode 4 
is selected (step S1026). If the cost C7 is smallest in step S1020, prediction is 
performed in the mode 3 adjacent to the mode 7 (step S1027). Then, a cost C3 
is calculated (step S1028) and the cost C3 is compared with the cost C7 (step 
S1029). Then, if the cost C3 is smaller than the cost C7, the mode 3 is 
selected (step S1031). If the cost C7 is smaller than the cost C3, the mode 7 is 
selected (step S 1030). 

[66] If steps S1016 and S1017 are performed, a smallest cost value among 
the costs CI, C6, and C8 is searched for (step S1032). If the cost CI is 
smallest, the mode 1 is selected (step S1033). If the cost>C6 is smallest, 
prediction is performed in the mode 4 adjacent to the mode 6 (step S1034) and 
a cost C4 is calculated (step S1035). Then, the cost C4 is compared with the 
cost C6 (step S1036). If the cost C4 is smaller than the cost C6, the mode 4 is 
selected (step S1038). If the cost C6 is smaller than the cost C4, the mode 6 is 
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selected (step S1037). If the cost C8 is smallest in step S1032, the mode 8 is 
selected (step S 1039). 

[67] If steps S1018 and S1019 are performed, a smallest cost value among 
the costs C2, C3, and C4 is searched for (step S1040). If the cost C2 is 
smallest, the mode 2 is selected (step S1041). If the cost C3 is smallest, 
prediction is performed in the mode 7 adjacent to the mode 3 (step S1042) and 
a cost C7 is calculated (step S1043). Then, the cost C3 is compared with the 
cost C7 (step 1044). If the cost C3 is smaller than the cost C7, the mode 3 is 
selected (step S1045). If the cost C7 is smaller than the cost C3, the mode 7 is 
selected (step S1046). If the cost C4 is smallest, predictions are performed in 
the modes 5 and 6 adjacent to the mode 4 (steps S1047 and S1048). Then, 
costs C6 and C5 are calculated (steps S1049 and S1050) and it is determined 
whether which cost among the costs C4, C5, and C6 is smallest (step S1051). 
If the cost C4 is smallest, the mode 4 is selected (step S1052). If the cost C5 is 
smallest, the mode 5 is selected (step S1053). If the cost C6 is smallest, the 
mode 6 is selected (step S1054). 

[68] That is, prediction modes are partitioned into several groups and an 
optimal mode is selected for each of the groups. Assuming that selection 
probabilities are the same in all prediction modes, 5.8 prediction counts are 
taken on the average until an optimal mode is selected. However, considering 
that the selection probabilities of the modes 0, 1, and 2 are higher, average 
prediction counts will be smaller than 5.8. 
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[69] FIG. 11 is a block diagram of a prediction mode determination 

apparatus according to an embodiment of the present invention. 

[70] The prediction mode determination apparatus includes a prediction unit 

1110, a prediction error calculator 1120, and a prediction error comparator 

1130. 

[71] The prediction unit 1110 priority-performs prediction for a 4x4, block 
in a mode with a highest probability to be selected. According to an 
embodiment of the present invention, the prediction unit 1110 can priority- 
perform prediction for a 4x4 block in the mode 0 (vertical mode), the mode 1 
(horizontal mode), and the mode 2 (DC mode). Also, the prediction unit 1 110 
can perform prediction for the 4x4 block in a mode selected according to 
mode information received from the prediction error comparator 1 130. 
[72] The prediction error calculator 1120 calculates and outputs prediction 
enrors (that is, costs) for prediction samples output from the prediction unit 
1110. The prediction error is a residue value obtained by subtracting a 
predicted value of a prediction target (for example, 4x4 block) from an 
original pixel value of the prediction target. A cost function used when 
calculating the prediction error may be one among the cost functions described 
above with reference to FIG. 8. 

[73] The prediction error comparator 1130 compares the cost values 
received from the prediction error calculator 1120, and selects and outputs a 
mode with a minimum cost value among the cost values. 
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[74] If mode information with the minimum cost value is transferred to the 
prediction unit 1110, the prediction unit 1110 performs secondary prediction 
according to the received mode information. Secondary prediction is 
performed in the modes 5 and 7, in the modes 8 and 6, or in the modes 3 and 4, 
as described above with reference to FIG. 8, FIG. 10A, and FIG. 10B. Then, 
costs in the corresponding modes are calculated by the prediction • error 
calculator 1120 and are compared by the prediction error comparator 1130. 
According to the compared result, a mode is selected. Next, the selected mode 
information is output as final mode information, or again transferred to the 
prediction unit 1110 when tertiary prediction is required. The prediction unit 
1110 performs tertiary prediction according to the received mode information. 
Tertiary prediction has been described above with reference to FIG. 10A and 
FIG. 10B. 

[75] In an encoder, prediction error samples are obtained in the prediction 
modes selected according to the above-described method, and transform and 
quantization for the prediction error samples are performed. Likewise, in a 
decoder, intra predictions are performed in the selected prediction modes. 
[76] As described above, according to the prediction mo4e determination 

method of the present invention, it is possible to select an optimal prediction 

j 

mode through the lower calculation times, thereby preventing an encoder from 
being complicated. 

[77] In other words, since a plurality of modes are partitioned into a 
predetermined number of groups, prediction errors are calculated and 
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compared for each of groups, and an optimal mode is selected, it is possible to 
reduce the number of used modes. Also, since predictions for a block are first 
performed in modes with priority (modes 0, 1, and 2), it is possible to prevent 
an encoder from being complicated. 

[78] The present invention may be embodied as a program on a computer 
readable medium including, but not limited to storage media, such as magnetic 
storage media (e.g., ROM's, floppy disks, hard disks, etc.), optically readable 
media (e.g., CD-ROMs, DVDs, etc.) and carrier waves (e.g., transmissions 
over the Internet). To be executed in an independent or present invention may 
be embodied as a distributed manner. 

[79] While the present invention has been particularly shown and described 
with reference to exemplary embodiments thereof, it will be understood by 
those of ordinary skill in the art that various changes in form and details may 
be made therein without departing from the spirit and scope of the present 
invention as defined by the following claims. 



20 



